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Abstract 

We generalize the notion of nuclear maps from functional analysis by defining nuclear ideals 
in tensored ^-categories. The motivation for this study came from attempts to generalize the 
structure of the category of relations to handle what might be called "probabilistic relations" . 
The compact closed structure associated with the category of relations does not generalize 
directly, instead one obtains nuclear ideals. 

Most tensored ^-categories have a large class of morphisms which behave as if they were part 
of a compact closed category, i.e. they allow one to transfer variables between the domain and the 
codomain. We introduce the notion of nuclear ideals to analyze these classes of morphisms. In 
compact closed tensored ^-categories, all morphisms are nuclear, and in the tensored *-category 
of Hilbert spaces, the nuclear morphisms are the Hilbert- Schmidt maps. 

We also introduce two new examples of tensored ^-categories, in which integration plays the 
role of composition. In the first, morphisms are a special class of distributions, which we call 
tame distributions. We also introduce a category of probabilistic relations. 

Finally, we extend the recent work of Joyal, Street and Verity on traced monoidal categories 
to this setting by introducing the notion of a trace ideal. We establish a close correspondence 
between nuclear ideals and trace ideals in a tensored ^-category, suggested by the correspondence 
between Hilbert-Schmidt operators and trace operators on a Hilbert space. 
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1 Introduction 



This paper develops a new categorical structure, called a nuclear ideal, which comes from two 
independent, seemingly unrelated, developments. These are Grothendieck's concept of nuclearity 
in functional analysis, see for example ||5^, and the usual notion of binary relations. The original 
motivation for this investigation was the need to generalize ordinary binary relations to probabilistic 
relations with an eye towards certain applications in computer science. However a satisfactory 
notion of what this generalization should be comes from the concept of nuclearity in functional 
analysis. This paper presents the new concept and gives several nontrivial examples of nuclear 
ideals. 

Relations form a basic and ubiquitous mathematical structure. There has been much activity 
in formulating what relations are "abstractly", so that one can generalize the concept to new 
situations. Typical examples of such formulations are the concept of cartesian bicategories |19] and 
allegories |^^. One of the key aspects of the category Rel is the fact that one has "transfer of 
variables" i.e. one can use the closed structure and the involution to move variables from "input" 
to "output". Intuitively speaking, this reflects the idea that the source and target of a binary 
relation are a matter of convention and a binary relation is an inherently symmetric object. In 
many situations that otherwise resemble relations, one finds that the closed structure does not exist 
and hence one loses the ability to transfer variables. A typical analogue of binary relations are the 
"probabilistic" binary relations, described at length later in the paper. Even in the absence of 
detailed definitions it ought to be clear that one cannot (indeed should not) rearrange the inputs 
and outputs of a probabilistic relation because there may be dependencies present among different 
inputs. What remains then in lieu of closed structure? We claim that it is precisely the nuclear 
ideals of the present paper. 

In these settings, there appears to be a tension between having identities and having compact 
closed structure. If one looks only at the nuclear ideal, one has a compact closed "category" without 
identities. On the other hand, the ambient category lacks closed structure. Others have observed 
that there are "categories without identities" , and given a wide range of examples and applications 



54]. However the interplay between the ideal and the ambient category is the point of the present 



work, not just the lack of identities. 

Another motivation for this work comes from considering Hilbert spaces. The tensored *- 
category of Hilbert spaces and bounded linear maps (hereafter denoted Hilb) shares much of the 
same structure as Rel. One of the goals of this paper is to measure the extent of this correspondence. 
Like the category of relations, Hilb has a tensor product and a tensor-preserving involution, which 
is the identity on objects. In the case of Hilb, it is given by the adjoint operation. However, 
the category of Hilbert spaces lacks the closed structure of Rel. The structure of Hilb has been 



axiomatized as the notion of a tensored ^-category ||2^, 23]. (In fact, it is a tensored C* -category, 
but we will not consider its normed structure here.) 

In this paper, we argue that a tensored *-category should be thought of as a category of (gener- 
alized) relations. The category of relations is compact closed, and this property is frequently taken 
to be fundamental in axiomatizing relational categories ]|l], However, the categories of relations 
which we consider are not compact closed, but rather contain a large class of morphisms, in fact an 
ideal, which has the basic structure of a compact closed category. To axiomatize this notion, we in- 
troduce the new notions of nuclear ideal and nuclear morphism. This idea is based on the definition 
of a nuclear morphism between Banach spaces, due to Grothendieck J33|, which was subsequently 
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axiomatized by Higgs and Rowe |^^. The concept of nuclearity in analysis can be viewed as de- 
scribing when one can think of hnear maps as matrices. Of course, in the finite-dimensional case 
one can always do this and it will be the case that all maps between finite-dimensional vector spaces 
are nuclear. The Higgs-Rowe theory applies only to autonomous (symmetric monoidal closed) cat- 
egories, while our definition applies to the somewhat different setting of tensored *-categories. In 
the case of a compact closed *-category, all morphisms are nuclear, while in Hilb with its usual 



tensored *-structure, the nuclear morphisms are precisely the Hilbert-Schmidt maps [42|. Note that 
since we are only considering Hilb with the L2 tensored *-structure, the notion of nuclear map we 
obtain is different from Grothendieck's notion arising from the category of Banach spaces (with, of 
course, the Li tensor product). 

A further goal of this paper is to introduce two new examples of tensored *-categories, in which 
integration plays the role of composition. The first such category is a category of generalized 
functions or distributions ^ Since a discrete relation on X x y can be viewed as a function 
/ : X xY {0,1}, it seems reasonable to model a "smeared out" relation as a continuous function 
f : U X V ^ R, where U and V are open subsets of Euclidean space. However, the identity 
for such a category would be the Dirac Delta which is not a function, but a distribution. We 
choose a particular class of distributions, the tame distributions, which are sufficiently functional 
to allow composition. We then present a nuclear ideal for this category. It will consist of the tame 
distributions with functional kernel. 

To build a category of probabilistic relations, one would like a category where the objects are 
probability spaces, and a morphism is a measure on the product space. The structure we eventually 
arrive at is the notion of conditional probability distribution, described in section 9. Categories of 



conditional probability distributions have previously been studied by Giry and Wendt [57, 58 1. 
Our formulation differs from theirs in that in our category, objects are equipped with measures and 
morphisms are measures on the product space satisfying an absolute continuity property. To each 
morphism, we are then able to associate a pair of conditional probability distributions. Again, in 
this case the nuclear ideal will consist of measures having a functional kernel. 

We also extend the recent work of Joyal, Street and Verity on traced monoidal categories to 
the present setting by introducing the notion of a trace ideal. For a given symmetric monoidal cat- 
egory, it is not generally the case that arbitrary endomorphisms can be assigned a trace. However, 
one can often find ideals on which a trace can be defined satisfying equations analogous to those 
of Joyal, Street and Verity. Our abstract definition is suggested by the usual trace construction 
in the category of Hilbert spaces, where there is a well-established relationship between maps in 
the trace class and Hilbert-Schmidt maps. In this case, we obtain the usual notion of trace of a 
bounded linear operator in the trace class. 

Acknowledgments- The authors would like to thank Mayer Alvo, John Baez, Michael Barr, Robin 
Cockett, Thomas Ehrhard, Jean- Yves Girard, Martin Hyland, Vojkan Jaksic, William Lawvere, 
Robert Seely and Michael Wendt for helpful discussions. We also received a number of extremely 
helpful comments from the anonymous referees. 
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2 Categorical Preliminaries 



We assume the reader is familiar with the notion of a symmetric monoidaQ category. A suitable 
reference is [^]. We now review some of the different closed structures such a category could have. 

Definition 2.1 A symmetric monoidal category is closed or autonomous if, for all objects A and 
B, there is an object A ^ B and an adjointness relation: 

Hom{A (S)B,C)^ Hom{B, A ^ C) 

The unit and counit of this adjunction are the familiar morphisms: 

ev: A {A ^ B) ^ B coev. A^ B ^ {A0 B) 

Examples of autonomous categories include the category of vector spaces and the category 
of relations. We obtain a pair of autonomous categories by considering Banach spaces. We can 
either consider Banoo, the category of Banach spaces and bounded linear maps, or we can consider 
the category Bani, of Banach spaces and maps of norm less than or equal to 1. In either case, 
the internal Hom is the Banach space of all bounded linear maps, and the tensor product is the 
completed projective tensor product |Q. 

Definition 2.2 A compact closed category is a symmetric monoidal category such that for each 
object A there exists a dual object A* ^ and canonical morphisms: 

V.I ^ A® A* 
'4):A* ®A^ I 

such that the usual adjunction equations hold: 

I® A 




together with the dual diagram for A* . In the case of a strict monoidal category, these equations 
reduce to the usual adjunction triangles. It is easy to see that a compact closed category is indeed 
closed and that A^ B = A* ®B. 

^We observe that the notation c: A® B ^ B ® Ais used for the symmetry, and I is used for the tensor unit. 
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Compact categories could also be defined as *-autonomous categories ||T^, |T5| with the additional 
canonical isomorphism A* 0B* = {A(SiB)*. *- Autonomous categories provide the basic framework 
for the model theory of the multiplicative fragment of linear logic pO| ]. 

We briefly describe the prototypical example, the category of relations. 

Definition 2.3 The category of relations, Rel, has sets as objects, a morphism from X to y will 
be a relation on X xY, with the usual relational composition. 

In what follows, X, Y, Z will denote sets, and x, y, z will denote elements. A binary relation on 
X xY will be denoted xTty. The identity relation will be denoted TP, and is defined as xTDx, for 
all X ^ X. Given a relation 1Z:X^Y, we let TZ:Y ^ X denote the converse relation. 

We verify that Rel is compact. The tensor product is given by taking the products of sets, 
and on morphisms, we have: 

TZ-.X ^Y S:X' ^Y' 
(x, x')lZ ® S{y, y') if and only if xTZy and x'Sy' 
The unit for the tensor is given by any one point set. We define the functor ( )*:Rel Rel 

by: 

X* = x n* = n 

The relation v: I ^ X ® X* is given by *v{x, x) for aW x ^ X and similarly for if). 



3 The Tensored *-Category of Hilbert Spaces 

Our notation for this section will be as follows. We will use brackets of the form (— , — ) to denote 
the inner product, which will be linear in the first variable. The associated norm will be denoted 
II — ||. If a is an element of the base field, then a will denote its conjugate. If is a Hilbert space, 
then H will denote the conjugate space. An orthonormal basis will be denoted {ej}jg/. A suitable 



reference for basic Hilbert space theory is |42| 



Let Hilb denote the category of Hilbert spaces and bounded linear maps, where "bounded" 
always means bounded in the norm associated to the inner product. We now discuss the structure 
of this category which is relevant to this paper. The first structure we need is the adjoint function 



Definition 3.1 Let 7i and /C be Hilbert spaces, and f:7i^fC& bounded linear map. Then the 
adjoint of /, denoted /*, is defined to be the unique bounded linear map /*:/C — > Ti. such that, for 
all a £ Ti, b £ /C, we have: 

(a,r(6)) = (/(a),6) 
Lemma 3.2 The adjoint construction satisfies the following properties: 

• {id-^y = id-^ 
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• /** = / 

• [f ® 9)* = f* ® 9* (The tensor product will be discussed below.) 

These conditions tell us that the adjoint operation provides a contravariant, tensor-preserving, 
involutive functor on Hilb which is the identity on objects. Given such a functor, it is clear that 
the category Hilb is much closer in its categorical structure to the category of relations than to 
the category of Banach spaces. 



3.1 Hilbert-Schmidt Maps 

We now discuss a crucial class of bounded linear maps, called the Hilbert-Schmidt maps. The 
material in this section can be found in [p^ j. 

Definition 3.3 If /: ^ /C is a bounded linear map, we call / a Hilbert-Schmidt map if the sum 

E 11/(^^)11' 

is finite for an orthonormal basis {ei}jg/. The sum is independent of basis chosen. 

It is straightforward to see that: 

Lemma 3.4 If f:7i ^ fC is a Hilbert-Schmidt map and g:TCi g'-K. fCi are arbitrary 

bounded linear maps, then g' f and fg are Hilbert-Schmidt. 

Thus the Hilbert-Schmidt operators on a space form a 2-sided ideal in the set of all bounded 
linear operators. A proof of the following theorem may be found in [42|. 

Theorem 3.5 Let ilSO(7i,IC) denote the set of Hilbert-Schmidt maps from TC to fC. Then 
YlSOi7i,lC) is a Hilbert space with: 

if, 9)= E (/(ei),ej>(e^-,5r(ei)) 

iG/jGJ 

Here, {ejjjg/ is an orthonormal basis for TC and {e^ jjej is an orthonormal basis for JC. 

3.2 The Tensor Product 

It is standard to construct the tensor product of Hilbert spaces 7i IC as the completion of the 
algebraic tensor product with respect to the inner product: 

{xi ®yi,X2 092) = {xi,X2){yi,y2) 

One then completes with respect to the L2 norm to obtain a Hilbert space. (Note that it is also 
possible to give an equivalent presentation that emphasizes the universal mapping property of the 
tensor. This involves the notion of a weak Hilbert-Schmidt mapping. This is explained in [42|, page 
132.) 
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Remark 3.6 We wish to emphasize that, in this paper, we will only be considering the L2 tensor 
product. Furthermore, the category Hilb will always be the category of Hilbert spaces with bounded 
linear maps, equipped with this tensored structure. 

Thus, our notion of nuclearity will not coincide with the notion obtained by viewing Hilbert 
spaces as Banach spaces and applying Grothendieck's definition, which of course uses the Li tensor. 

For us, the most important property of the Hilbert tensor product is its relation to Hilbert- 



Schmidt maps. This is given by the following theorem |42], p. 142: 



Theorem 3.7 We define a linear mapping JJ-.Ti ® K, ^ HSO('H,/C) by U{x®y){u) = {x,u)y, 
where x ® y £ 7i ® KL. Then U is a unitary transformation of 7i ® fC onto HSO('H,/C). In 
particular, we note that the morphism U is a linear bisection. 



4 Tensored *-categories 

The category Hilb, of Hilbert spaces and bounded linear maps, shares many of the properties of a 
compact closed category, except for the closed structure. Hilb is in fact an example of a tensored 
*- category. We now develop this theory. 

Definition 4.1 A category C is a ^-category if it is equipped with a functor (— )*: — > C, which is 
strictly involutive and the identity on objects. (Note that the strict involution may be replaced with 
a coherent involution, but we will not require that level of generality.) A *-category is tensored if 
it is symmetric monoidal, (/ (8> g)* = f* ®g* , and there is a covariant conjugate functor, ( ): C ^ C, 
which commutes with the *-functor and has natural isomorphisms: 

• A = A (We will generally take this to be an equality.) 

• A® B = A® B (We will generally take this to be an equality.) 

• 1^1. 



satisfying the usual monoidal equations, and the following equation. Suppose that f: I ^ I. 

f* 




In all of our examples except for those involving complex Hilbert spaces, conjugation will simply 
be taken to be the identity. In this case, the previous diagram implies that if /: I — > I, then /* = /. 



The notion of a tensored *-category is the first step towards defining a tensored C* -category, or 
multiobject C*-algebra |29, 23]. This theory has been developed quite extensively in the previously 
cited references. Among the results established is a representation theorem stating that such 
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categories have faithful structure-preserving embeddings in Hilb. This should be thought of as a 
multiobject version of the Gelfand-Naimark-Segal theorem. 



Examples of tensored *-categories 

• Rel 

• Hilb 

• Hilbj^;, the category of finite-dimensional Hilbert spaces. 

• URep(G), the category of unitary representations of a compact group G. 

• URepj^(G), the category of finite-dimensional unitary representations of a compact group 
G. 



Further examples can be found in [29, Note that examples 2 and 4 are tensored =f=-categories 
which are not closed. We will present other examples of tensored *-categories which are not closed. 

Even though tensored *-categories are not compact closed, they share much of the same struc- 
ture. One of the goals of this paper is to introduce a structure for measuring the extent to which 
such a category is closed. 

5 Nuclearity 

One of the characteristic features of compact closed categories is the ability to distribute the dual 
functor across the tensor product. This is represented by the equation {A B)° = A° ® B° . {A° 
denotes the dual object. We temporarily adopt this notation to avoid confusion with the *-functor 
we will be discussing later. In the context of tensored *-categories, one should keep in mind the 
equation A° = A*.) This allows one to arbitrarily repartition the morphism or "interface" in the 
terminology of interaction categories Q. The categories we will encounter typically allow such 
repartitioning for some maps, but do not meet all the requirements of being a compact closed 
category. 

We now introduce the related notion of nuclearity in a symmetric monoidal closed category, 
due to Rowe |5C], and subsequently studied by Rowe and Higgs [|^]. The idea is suggested by 



Grothendieck's work on topological tensor products and nuclear spaces [^. Grothendieck defined 
a continuous linear map f: A ^ B between Banach spaces to be nuclear if it can be written as 
/(a) = T,fi{a)bi where S/j ® 6j is an element of the completed projective tensor product A° B. 
We begin by noting that in any symmetric monoidal closed category, there is a morphism of the 
form: 

(f:B(g)A°^A^B 

Here, A° = A ^ I, where / is the unit for the tensor. 
This is constructed as the transpose of the composite: 

B A° A'^ B I ^ B 
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Definition 5.1 Let C be a symmetric monoidal closed category. Let ip denote the canonical mor- 
piiism ip:B<^A°^A^B.lif:A^Bva.C, tiien let n{f): / ^ A ^ i? be the name of /. We 
say that / is nuclear if there exists p{f)- 1 ^ B A° such that the following diagram commutes: 




A^ B 



We will refer to p{f) as a pseudoname for /. (We should point out that there are some cases in 
which a pseudoname is not unique.) We say that an object of C is nuclear if its identity map is 
nuclear. 



Lemma 5.2 Suppose that f: A ^ B and g:C ^ D are nuclear, then so are: 

• f°:B° A° 

• /'/: A ^ E for any morphism f':B^E 

• fh: F ^ B for any morphism h: F ^ A 

• f(^g:A(g)C^B(g)D 

All of the above can be obtained by straightforward diagram chasing. For example, in the third 
item, one can choose p{fh) = p{f); {id (8) h°). It is not in general the case that if f,g are nuclear, 
then so is / ^ g. However, if C is *-autonomous with unit as dualizing object, then f —o g will 
also be nuclear, [^] p. 70. 

In a compact closed category, the map ip is an isomorphism, and thus every map is nuclear. 
Furthermore, we can see the following: 

Theorem 5.3 ( [p7|] , Thm. 2.5) For an arbitrary object A in C, a symmetric monoidal closed 
category, the following are equivalent. 

• A is nuclear 

• The morphism ip: A ® A° ^ A ^ A is an isomorphism. 

• The morphism ip: B A° ^ A —o B is an isomorphism, for arbitrary objects B. 



Theorem 5.4 For any symmetric monoidal closed category, the full subcategory of nuclear objects 
is compact-closed. 
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Proof. Suppose that A is a nuclear object. Then, choosing a pseudoname for the identity gives a 
morphism of the form I ^ A® A° . It only remains to show that the adjunction triangles commute. 
We will consider one of the two adjunction triangles. 



p{id) id 



n{id) ® id 



{A 




id ® ev 



It is standard that the lower leg of the diagram is the identity. The upper leg of the diagram 
corresponds to the adjunction triangle. The upper triangle in the above square is the definition of 
pseudoname. The lower triangle is a straightforward exercise. 



In Banoo or Bani, we recover Grothendieck's original definition of nuclearity. The nuclear 
objects are the finite-dimensional Banach spaces. In the category of vector spaces, a morphism 
is nuclear if and only if its image is finite-dimensional. Again, a vector space is nuclear if and 



only if it is finite-dimensional. In |37], the authors explore the notion of nuclearity in the category 
of complete join semilattices CJSL. It is well known that this is a symmetric monoidal closed 



category, in fact *-autonomous |41|. The authors completely characterize nuclearity in this case 
(This result is closely related to Raney's notion of a tight morphism [^].): 

Theorem 5.5 (Higgs, Rowe) A morphism f:A^B in CJSL is nuclear if and only if there exists 
g: B ^ A such that for all a £ A, f[a) = sup{h\a ^ g{b)}- An object is nuclear if and only if it is 
completely distributive. 



Remark 5.6 Following recent work of Joyal, Street and Verity on traced monoidal categories, 
one can now observe that, in a symmetric monoidal closed category, it is possible to define a trace on 
the nuclear morphisms as follows, under the assumption that pseudonames are unique. If f: A ^ A 
is nuclear, then tr{f):I I is given by: 

tr{f) = p{f)]ev: I ^ A0A° ^ I 

where ev: A ^ A° ^ I is the usual evaluation map. Then given h: A ^ B a nuclear map, and 
g:B ^ A arbitrary, one can verify the usual trace equation tr[gh) = tr[hg). This is seen by the 
following diagram: 
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The righthand diamond is the usual (di)naturality of evaluation. The two triangles on the left 
are the equations for p{gh) andp{hg). 

While this theory is satisfactory when considering symmetric monoidal closed categories, there 
are nonclosed categories which exhibit similar structure. For example, the category of Hilbert 
spaces is not closed, but the class of Hilbert-Schmidt maps seem to have something like a nuclearity 
property. We will soon exhibit other such categories. One of the goals of this paper is to extend 
the above notions to a larger class of categories, specifically to *-categories. We now introduce a 
new notion, that of a nuclear ideal. 

Definition 5.7 Let C be a tensored *-category. A nuclear ideal for C consists of the following 
structure: 

• For all objects A, B e C, a subset J\f{A, B) C Hom{A, B). We will refer to the union of these 

subsets as J\f{C) or A^. We will refer to the elements of as nuclear maps. The class A^ must 
be closed under composition with arbitrary C-morphisms, closed under 0, closed under ()*, 
and the conjugate functor. 

• A bijection 6:J\f{A, B) —>■ Hom{I, AiSi B). U f: A B is a nuclear morphism, note that we 
can use the bijection 9 and the *-functor to construct morphisms of the form: 

1. e{f):I ^A(g> B 

2. e{f)*:A(^B^ I 

3. e{f*):I ^B(^A 

4. e{f*)*:B(^A^ I 

We shall frequently refer to these morphisms as transposes of /. It will always be clear 
from the context which transpose is being considered. The bijection 9 must also satisfy the 
following properties: 

1. Preservation of tensored *-structure The bijection 9 must preserve all of the ten- 
sored *-structure. In other words. 



11 



(a) U f : A ^ B and g : C ^ D are nuclear, then 9{f ^ g) = 9{f) 6{g). More 
precisely, the map 9{f ^ g): I ^ A (Si C B D is given by the composite: 



I^I(g)I^A(E)B(g)C(g)D^A(g)C(g)B(E)D = A(E)C(g)B(g)D 

Furthermore, the transposes of a map of the form f: I ^ A are given by composition 
with the evident isomorphism. 

(b) 9{f) = 9{f*) = 9{f). Again, more precisely, we would say: 



9{f) = co9{n = 0{f)°i 
where c is the symmetry and t is the isomorphism l: I ^ I. 
2. Naturality For any f: A ^ C and g. B ^ D, the following diagram commutes: 



M{A, B) Hom{I, A(g)B) 



Hom{I, f ® g) 



N{C, D) Hom{I, C0D) 

Note that since the class of nuclear morphisms is closed under composition with arbitrary 
C-morphisms, the function M{f*,g) is well defined. 

3. Compactness Let f: A ^ B and g: B ^ C he nuclear. 
Then the following should commute. 



A 



I^A 



e{g) id A 



B®C ® A 



gf 



c 



C®I 



idc®e{rY 



C(SB®A 



This completes the definition of nuclear ideal. In the case where A is a nuclear object and 
f = g = id A, then this last equation reduces to the usual adjunction equation for a compact closed 
category. We will see that it is also related to the "yanking" axiom of 



Given a category C and a nuclear ideal A^, we say that an object ^ of C is A^- nuclear if we 
have that ^/{A,—) = Hom{A, —). Note that by the ideal property, this is equivalent to saying 
that the identity map for A is nuclear. Typically, this notion of nuclear object is capturing the 
"finite-dimensional" subcategory. It should not be thought of as describing Grothendieck's much 
richer theory of nuclear spaces. 

Note that we are not claiming that the transposition map is in any way unique; different choices 
of 9 could conceivably give different nuclear ideal structures. The usual uniqueness arguments, see 
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for example pp. 80-82, do not apply here in that we may not transpose the identity map. 
Thus it is possible that several distinct nuclear structures may exist on a given category. We are 
still pursuing this question. However, we know of no such examples. In the examples presented 
in this paper, the choice of the transpose is obvious and canonical, given the structures under 
consideration. 

One of the consequences of the above definition is the "sliding" equation of Joyal, Street and 



Verity |0|: 



Lemma 5.8 Suppose f: A ^ B and g: B 

for any nuclear ideal: 



A are nuclear. Then the following diagram commutes 



A(g)B 






B®A 



0if*r 



This equation is a straightforward consequence of the axioms. We will see in Section 8 that it 
corresponds to the familiar trace equation tr{fg) = tr{gf). 

Theorem 5.9 Let {C,M) be a nuclear ideal for which all objects are nuclear, then C is a compact- 
closed category. 



Proof. If A is an object of C, then the transpose of the identity will be a morphism of the form 
I ^ A f^i A. The commutativity of the adjunction triangles follows from the compactness require- 
ment of the definition. 



Theorem 5.10 The set of Hilbert- Schmidt maps forms a nuclear ideal for Hilb. 

Proof. Let 7i and /C be Hilbert spaces, and let M(Ti.,IC) be the set of all Hilbert-Schmidt maps 
from Ti. to /C. It is evident that Hom{I,TC ® IC) = Ti ® KL. So the morphism [/, defined in 3.7, 
will act as a transpose operator. We saw in section 3 that this map was a linear bijection. It only 
remains to check the equations. These are a straightforward consequence of linearity and properties 
of the adjointness operator. 
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The nuclear objects in this case are precisely the finite-dimensional Hilbert spaces. Thus we 
recover the familiar compact closed subcategory. The same program can be carried out for categories 
of representations such as URep(G). 



5.1 Partial Injective Functions 

Define a category PInj as follows. Its objects will be sets, and morphisms will be partial injective 
functions, that is to say partial functions which are monomorphic when restricted to the domain. 
These partial functions were used by Danos in his modeling of the geometry of interaction ||2T[ . 

If /:X — > y is a morphism, let Dom(f) be its domain, i.e. Dom(f) = {x £ X\f(x) is defined}. 
This category has an evident *-structure, and if we choose the cartesian product of sets as a tensor, 
then we evidently have a tensored *-category. We now demonstrate that this category has an 
evident nuclear ideal. Define: 

J\f{X,Y) = {f:X ^ Y\Dom{f) has cardinality or 1} 

Then one can see that we have an obvious bijection between Hom{I, X f^iY) and M{X, Y). 

Theorem 5.11 The above construction defines a nuclear ideal for Fin] . 



5.2 Crossed M-Sets 

The following is based on Freyd and Yetter's notion of a crossed G-Set, which they use in their 



work on braided compact closed categories [28|. In this paper, we will only consider a commutative 
monoid, which gives a symmetric monoidal category. We hope to explore the nonsymmetric and 
braided versions of this construction in future work, as well as the connections to topological 



quantum field theory [10|. 



Definition 5.12 Let M be a commutative monoid with identity e. Define a crossed M-set to be 
a (left) M-set X, together with a function | | : X — > M such that \mx\ = \x\. (This formula 
is more complicated in the nonabelian case. With a nonabelian group, we would require that 
\gx\ = g-'^\x\g.) 

Now define a category XRel as follows. Objects are crossed M-sets, and maps are relations 
R:X^Y such that: 



• xRy =^ mxRmy 

• xRy =^ |x| = \y\ 

Freyd and Yetter construct a category where the objects are functions satisfying precisely these 
requirements. They use a nonabelian group and the braiding is the symmetry adjusted appropri- 
ately by the action of G. They then use this category to develop knot invariants pq|. In subsequent 



work, Yetter uses crossed G-sets to construct topological quantum field theories ||5^. See also [47| 
If X and Y are crossed M-sets, define X (8) Y as cartesian product with componentwise action 
and |(x,y)| = The unit is the one element set / = {*}. Define | * | = e. 

Theorem 5.13 XRel is a tensored ^-category. 
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Note that XRel is not compact. The counit of the adjunction would be required to satisfy 
*R{x,x) for all x G X, but this would hold if and only if |xp = e. This will be our definition of 
nuclear object. 

Now for all X, Y, define AA(X, Y) C Hom{X, Y) by: 

i? : X — > y is nuclear if and only if xRy =^ \x\'^ = = e 



Theorem 5.14 This defines a nuclear ideal for XRel. 



6 Distributions as Relations 

In this section, we introduce a generalized category of relations based on the idea of distributions. 
The guiding intuition is that composition should be determined by an integral of the form: 

fix, y); ipiy, ^) = j z)dy. 

The viewpoint here is that the notion of integration generalizes the existential quantification that 
appears in the definition of relational composition. We will refer to this formula as the "convolution 
formula." We now introduce a framework in which this makes sense. A naive approach is to view 
ip{x^ y) and ^(y, z) as real-valued functions. However, for such a "category" to have identities would 
require an equation of the form: 



(p{x,y)6{y,y')dy = ip{x,y') 
and similarly for left composition. The "function" playing this role is in fact the Dirac 5 which 



is not a function but a generalized function or distribution in the sense of Schwartz [52, 56, 
Unfortunately multiplication of distributions is not always well-defined. Formulas like the one 
above are sensible only for certain limited kinds of distributions. In the rest of this section, we 
review basic facts about distributions and then develop a theory of what we call "tame" distributions 
for which the above integral formula makes sense. 

Tame distributions are mentioned in the extant literature (see, for example, Dieudonne's "Trea- 
tise on Analysis", volume 7, chapter 23, sections 9 and 10 (2^), but are not given a name. 



6.1 Basics of Distributions 

Let denote a nonempty open subset of R". Let denote the set of C°° (smooth) functions on 
and D(r2) denote the smooth (complex- valued) functions of compact support on $7. We will refer 
to the elements of D^i}) as test functions. In what follows, we use Greek letters such as (p, ip, rj as 
test functions. V^Q) is given the structure of a topological vector space as follows. This structure 
is described for example in 

We begin by considering a compact subset C 0, and letting D(r2; K) be the set of continuous 
functionals on with support contained in K. Then we define a family of seminorms on T>{Q; K) 
by the following formula, where Q^r--^" denotes the partial derivative with respect to the listed 
variables: 
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i 3 

\Lp\m = SMp{|9^i'"^"99(x)|: X G K and i + . . .+ j <m] 

We then give T)[Q.\ K) the least topology such that each of these seminorms is continuous. The 
existence of such a topology is proved on page 12 of With this topology, K) is a Frechet 
space, i.e. it is locally convex, metrizable and complete. 

Now observe that 

V{n) = [j{V{n; K)\ K onandK is compact} 

We then give 15(0) the finest locally convex topology such that the inclusions 15(0; K) C 15(0) 
are continuous for every compact K. This is known as the inductive limit of the topologies on 
V{n;K). 

Theorem 6.1 (p. 25 ^) The topology that inherits as a subspace of'D{^) is the same as 

its original topology for every compact K . A linear functional on Di^D.) is continuous if and only if 
the restriction to D^^l-jK) is continuous for every compact K. 

With this topology, ViVt) is not metrizable. However it is an LF space (locally Frechet) in the 
sense of p. 126. As such, it is locally convex, Hausdorff and complete. 

Then we define a distribution on 17 to be a continuous, linear (complex-valued) functional on 
Let T>'[^) denote the set of all distributions on $7. Let V{Vl) be given the weak topology, 
p. 45 [^] or p. 197 [^6|. This is equivalent to the topology of pointwise convergence, and is 
locally convex, Hausdorff and complete. We will also have need of the following extension theorem 
II p.39. 

Theorem 6.2 Let E, F be two Hausdorff topological vector spaces, with A a dense subset of E and 
f a continuous linear mapping of A into F. If F is complete, then there is a unique continuous 
linear mapping f from E into F which extends f . 



We now describe some examples. 



1. Let \-ioc{^) denote the space of locally integrable functions. Suppose that / € LiociP). Define 
a distribution Tf by: 



Tf{'^) = j^f{x)^{x)dx 

Note that two locally integrable functions determine the same distribution if and only if they 
are equal almost everywhere ||56[1 . A distribution of this form is called regular, and the function 
/ is called the kernel of the distribution. A distribution which does not arise in this way is 
called singular. Regular distributions are fundamental examples, in fact there are a number 



of strong results regarding the approximation of distributions by regular distributions |56|. 
This justifies thinking of distributions as generalized functions. 
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2. As a special case of the previous example, we observe that every test function is itself locally 
integrable, and so induces a regular distribution. Thus we have a canonical inclusion 

l:V{X) ^ V'{X) 

given as follows: 

(t){x) ^ [ip{x) £ V{X) ^ J (j){x)ijj{x)dx] 

There are similar inclusions for the set of locally integrable functions or smooth functions. 

3. For any point x G 0, let 6x{<^) = ^{x). If G O, we denote 5q simply as 5 and refer to 
it as the (one-variable) Dirac delta. One can show that this distribution is singular, see for 
example ^, |5^ . 

4. If C R, we may also "differentiate" the previous distribution via the formula: 

6U^) = -ip'ix) 

This distribution is also singular. More generally, if 17 C R" and T £ we have the 

formulas: 

A(r)(^) = -r( 

OXi OXi 



94-^i(T)((^) = (-l)'+-+-'r(5^i-^"(v7)) 

These formulas allow one to "differentiate" nondifferentiable functions, and are one of the 
many advantages of distributions. See, for example, Chapter 2.3. 

5. When considering 17 x fi, we have the trace distribution, p9| Example 5.2.2, given by: 



LP G 'D{Q. xVL)^ (p{x, x) 
6.2 The Schwartz kernel theorem 

One is often interested in distributions on product spaces, especially in the theory of differential 
equations and their associated Green's functions. In this situation the analogy between distributions 
and "infinite-dimensional matrices" is quite striking. The theory of kernel distributions can be seen 
as a formalization of this analogy. In the analysis literature, the notion of "kernel distribution" is 



studied at length, see for example the massive treatise of Dieudonne [^] or the book by Treves |56|. 
When considering a space of test functions of the form T>(X x Y), there is a canonical subspace 
of fundamental importance. Consider the tensor product "D^X) (8) DiY). A typical element of this 
space is of the form 
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n 

ifi Tpi where (pi G T>{X) and ipi G I^(l^) 

There is a canonical inclusion of T>{X) ^^(^) into D(X x Y) given by: 

V ^ [(x, y) (/7(x)V'(y)] 
The result we will have use for is: 
Proposition 6.3 The space ^{X) (g) ^^{Y) is sequentially dense in V{X x Y). 

Now we have a chance of defining functions on 'D{X x Y) as the unique continuous extension 
of functions defined on T>{X) (8) 'D{Y) using Theorem |6.2| . 

One of the fundamental results in the theory of distributions is the Schwartz kernel theorem, 
which gives conditions under which maps from to T>'(Y) can be realized as distributions on 

X xY. We need the following notations to state the theorem. If / is a distribution on X x y and 
(f) S T^{X) then /*((/>) will be the function from "DiY) to the base field given by G ^(^) "-^ fi'P®^) 
and f*{ip) is given by the evident "transpose" formula. We have not yet said that /*((/>) and f*{ip) 
are distributions; that is part of the content of the kernel theorem. 

The Schwartz kernel theorem states: 

Theorem 6.4 Let X and Y he two open subsets of R" and R"*. 

1. Let f be a distribution on X x Y . For all functions (p G ^(^) the linear map f*{4>) is a 
distribution on Y . Furthermore, the map (f) f *{(/>) from to D'iY) is continuous, 
when D'iY) is given the weak topology. 

2. Let be a continuous linear map from T>[X) to T>'(Y) . Then there exists a unique distribution 
on X xY such that for (j) G 'T){X) and ij) G 'F'iY) the following holds: 

fi^^i^) = f*{<t>m 

Evidently, by symmetry, the same result applies for /*. In light of the kernel theorem, we may 
now state the following definition. 

Definition 6.5 Suppose that f is a distribution on X xY , then we obtain the following continuous 
maps (supposing that (p G T>{X),ip G T^lY) are arbitrary): 

L f,:V{X) ^ V'{Y) IS given by /*(</>)(V^) = /(<A® V') 

2. f*:V{Y) V'{X) is given by f*{ip){cp) = f{^ ® V) 



18 



6.3 Tame Distributions 

To pass from the "discrete" category of ordinary relations to a category of "continuously varying" 
relations, we should replace the usual notion of morphism in Rel, a function X x y ^ 2, with an 
integrable function X x y — > R, where X and Y are now open subsets of some Euclidean space. We 
have already seen, however, that functions do not suffice. One must pass to a class of generalized 
functions or distributions. While distributions satisfy many properties of functions, they cannot be 
multiplied and hence the composition formula that we had proposed does not make sense. Thus 
our goal is to introduce a class of distributions which are sufficiently "functional" as to allow us to 
compose them using the integral formula discussed above. 

We will use a notion defined by Dieudonne in ||2^ . It will provide the first step towards defining 
a composable class of distributions. Note that £{X) is the space of all smooth complex-valued 
functions on X (not necessarily of compact support). Unfortunately, Dieudonne uses the term 
regular which conflicts with the terminology above. We therefore use the term Dieudonne-regular. 

Definition 6.6 We say that a distribution / G T>'{X x Y) is Dieudonne-regular if 

1. For all functions (j) £ ^i^)? f*{4') is in £-iX)-, that is to say there exists (j) G £{Y) such that 
the distribution /*((/>) G T>'{Y) is defined by: 

JY 

2. Similarly, for all functions ip £ ^{Y), f*{ip) is in £{X). 

An equivalent statement is that the function f^, : T^iX) D'^Y) specified by the kernel theorem 
factors through the inclusion £(Y) ^ and similarly for /*. 

We would like to define our composition as follows. Given distributions / G D(X x Y),g G 
'D{Y X Z) which are Dieudonne-regular, we try to define a distribution f;g& V{X x Z) using the 
following formula (with G 'D{X),^ G P(Z)). 

JY 

Here (p is the element of £{Y) associated to the distribution f*{4>), and 7 is the element of £{Y) 
associated to the distribution /*(7). 

However, the above integral may well be infinite. Thus we must add an additional assumption 
which assures the finiteness of this integral. One possibility is to require not only that the two 
kernels be smooth, but that they have compact support.^ Thus, we have the following: 

Definition 6.7 A tame distribution on X x y is a distribution / on X x y such that each of /* 
and /* factor continuously through the appropriate l, where t is the inclusion of the space of test 
functions into the space of distributions. Explicitly, there exist continuous linear maps 

fL:V{X) ^ V{Y) 

^In fact, one could use a more general class of functions, such as the square integrable functions, but we prefer 
the symmetry of the present definition. 
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fR:V{Y)^V{X) 
such that for every (p £ T^{X) and V ^ ^0^)? have: 



fMW = rWl^) = f{<t^®^) = j fLi4>Wy = j <PfRWdx 

Note that we are not saying that and have functional kernels and certainly not that / 
has a functional kernel. But rather that /* and its adjoint /=„ map test functions to distributions 
with test functions as kernels. In some sense, tame distributions are allowed to be mildly singular, 
in that composing with a test function "tames" the singularity. 



Dieudonne, in [^, page 77, examines the question of when the operators /* and /=„ map test 



functions to test functions, and he derives the following theorem. 

Theorem 6.8 Let f be a Dieudonne-regular distribution on X x Y . The following are equivalent: 

1. The operator /* extends to a continuous linear map from the Frechet space £{X) to the 
Frechet space ^^(5^)- 

2. The operator f* maps T>{Y) to V{X). 

3. The operator f* maps £'(Y) to £'{X), where £'{Y) is the space of distributions of compact 
support (see ^ for the definition of support of a distribution). 



6.4 Examples 

• Let X be an open subset of R". The trace distribution on X x X is given by Tr{rj) = 
J r]{x,x)dx where ri(x,x') £ T>{X x X). From this definition it follows that Tr^{(j)){ip) = 
Tr*{ilj){(f)) = Tr((f) "0) = / 4>{x)ip{x)dx. Thus we clearly have Tri{(f)) = Trfi{(j)) = <j), which 
shows that 5 is tame. This tame distribution will act as the identity in our category. 

• Suppose that T is a regular distribution on X x y with a test function y) as its kernel, 
that is to say: 

T{a{x,y)) = (3{x,y)a{x,y) 
JxxY 

Then T is tame with its associated functions being given by: 

Tl{cP)= I (i{x,y)m 
Jx 

Tn{iP) = J^f3{x,y)ij{y) 
We write T{X, Y) for the tame distributions on X x y. 
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6.5 Composing tame distributions 



Given tame distributions we can define the following operation which will serve as composition. 
Suppose that / G r{X,Y),g e T{Y,Z). We define f;g £ r{X,Z) as follows. Given that / is 
tame, we have a continuous function fi : T)[X) !){¥). Applying the first part of the Schwartz 
kernel theorem to g, we obtain a morphism g^, : T^lY) — > Composition gives a continuous 

map 'D{X) D^Z). By the second part of the kernel theorem, we obtain a distribution on X x Z. 



Alternatively, we could use the extension theorem, Theorem 6^. Let <p E € T^{Z). We 

set 

{f;g){^0^)= f fL{ct>)gRWdy. 



This, of course, only defines f;g on T^lX) (gi T){Z) rather than on T){X x Z). We then use the 
fact that the tensor product is a dense subspace to extend composition to all of T>{X x Z). One 
observes that /; g is tame as can be seen by an elementary calculation, noting (/; g)i = fi; gi and 
(/; 9)r = gR] Ir and the tameness of / and g. 



6.6 The category DRel 

Definition 6.9 The category DRel has as objects open subsets on R", and, as morphisms, tame 
distributions. Gomposition is as described above. 

Theorem 6.10 DRel is a tensored ^-category. 

Proof. Evidently we can verify properties of the composition /; g by carrying out calculations on 
the distribution defined on 'D{X)®T){Z) and appealing to continuity and the density oiT){X)®'D{Z) 
in T){X X Z). We have already noted above that f]g is tame. A simple calculation shows that the 
trace distribution is the identity for composition. 

To verify associativity we calculate as follows. Let / G P(X x y), g G T>{Y x Z) and h G 
T){Z X W) be tame distributions. Then we have: 



{{f-g)-h){<P{x)®p{w)) = I if;g)L{<P)hRip)dz 

9L{fL{(t>))hR{p)dz 



fL{4')9R{hR{p))dy 
= {f-{g;h)){^®p) 

Thus we have shown that DRel is a category. The tensor product is given as follows. 
Given objects X and Y we define X ®Y SkS the cartesian product space X x Y . Given mor- 
phisms in DRel f:X^Y and g:X' Y' we can define f ® g:X ® X' Y ®Y' as fol- 
lows. We first define f g as a distribution on (gi 'D{X') 'D{Y) (g) 'D{Y') by the formula 
(/ ® 9){4>{x) ® 4>'{x') (g) tp{y) tp'{y')) = /{(p ^5 '>P)gi4>' (g V'O- It is routine to verify that this is tame. 
We extend / (g 5 to all of V{X x X' x Y x Y') as above. The one-point space, written / = {*}, is 
the unit for the tensor (with measure p{{*}) = 1). 

Finally the *-structure is the identity on objects. On morphisms, the only thing that changes 
is the role of fi and /r. The conjugate functor is taken to be the identity. 
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Remark 6.11 ^45 an example, we will describe Hom{I,X), where X is an arbitrary object. 
Clearly, is isomorphic to the base field. We must have two functions: 



fL:V{I)^V{X) 



fR:V{X)^V{I) 



such that, for all ip G 'D{X): 



l^fLil)4' = l^lfRW 



But evidently Jj lfji{'ip) = fR{Tp). So the function fR is uniquely determined by the function f^. 
Hence we may conclude that Hom{I , X) is in bijective correspondence to test functions on X. 

We now display a nuclear ideal for DRel. We remarked that not all tame distributions can 
be viewed as integral operators with functions as kernels. In particular the identity morphisms do 
not have this property. However, we will see that tame distributions with functional kernels form 
a nuclear ideal. 

Definition 6.12 Given objects Y and Z of DRel we define the set of nuclear morphisms, written 
M{Y, Z), as the collection of tame distributions g:Y ^ Z such that 3/3(y, z) £ V{Y x Z) with the 
property that for every (j){y, z) G T){Y x Z): 



Note that the test function (3{y, z) G ViY x Z) associated to to the tame distribution g is 
unique. Thus, the set N{Y, Z) is in bijective correspondence to VlY x Z). 

Theorem 6.13 The sets N(Y, Z) form a nuclear ideal for DRel. 

Proof. As already remarked, if 5 G NiY, Z) and if (3 is its kernel, then: 



To verify that we have an ideal, we have to show that for any / G T{X, Y) the composite /; g 
is nuclear and symmetrically for composition on the other side of In order to verify this we need 
to find a kernel for f;g. We claim that this kernel is a{x,z) =(if fniPiy, z)) where we interpret 
this formula as follows. For each fixed z £ Z (3{y, z) is a smooth function of compact support in 
Y; fR acts on this function to produce a function of compact support in X. The function a{x, z) 
evidently has compact support, and its smoothness is a consequence of the continuity of fR. It 
suffices to prove this for functions (5 of the form z) = a{y)b{z) where a G 'D{Y) and b G 'D{Z). 
This follows from Proposition |6.3| which implies that arbitrary (3 can be written: 





P{y,z) = lim ^ ai^niy)bi,niz) 



i=l 



The general result then follows from the linearity and continuity of fR. 
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Now observe that for a fixed z: 



fR{Piy,z)) = fR{aiy)b{z)) = fR{a){x)b{z) 

Now we calculate as follows, again letting f3{y,z) = a{y)h{z) and relying on linearity and 
continuity for the general result: 



U;g){<l){x)®^{z)) = I fLmy)gRmy)dy 

fLmy)[f I3{y,z)^{z)dz]dy 

JZ 

fL{(t)){y)[3{y, z)dy]'4){z)dz 



Y 



Y 



Z JY 

fL{<P){y)a{y)b{z)dy]tl;{z)dz 
zJy 



Z JY 



fL{(t>){y)a{y)dy]b{z)tlj{z)dz 
(t){x)fR{a){x)dx\b{z)ip{z)dz 
(l){x)[j[fR{a){x)b{z)]^l,{z)dz]dx 
(j){x)a{x, z)ip{z)dxdz. 



z Jx 



X 



X Jz 



It follows that /; g' is an integral operator with a as its kernel. The verification for composition 
on the other side of g is very similar. 

To complete the proof, we need to show that Hom(I,X (giY) = J\f{X,Y). This isomorphism 
is described in Remark 6.11 . It remains to verify the equations. Naturality requires an argument 
similar to the previous calculation. Compactness is quite straightforward. 



7 The Category PRel 

In this section, we define a category of probabilistic relations, and describe a nuclear ideal for it. We 
will see that we indeed get most of the important properties of the category of relations, i.e. we have 
a tensored *-category with a nuclear ideal. Thus one may think of this category as representing 
relations "smeared out probabilistically". Once again, as in DRel we have a situation where the 
identity maps are too singular to be in the nuclear ideal. The nuclear ideal can be thought of as 
functions but the ambient category has to be described in terms of measures. 

7.1 Basic Definitions of Measure Theory 

We assume the reader is familiar with the basic concepts of measure theory. We recall the basic 
definitions for completeness. A reader who remembers these definitions can skip to the start of the 
next section without loss of continuity. 
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Definition 7.1 A a -field S on a set X is a collection of subsets of X which 

1. includes the whole space X, 

2. is closed under complementation, and 

3. is closed under finite and countable unions. 

A measurable space is a set together with a cj-field. A measurable function from a measurable space 
{X, Sx) to (y, Sy) is a function from XtoY such that for all B € Sy we have f^^{B) £ Sx- 

Given a measurable space (X, Sx), we call the members of Sx measurable sets. If is a 
measurable set then the characteristic function of B is denoted xb and is clearly measurable. 

Definition 7.2 A measure on a measurable space {X, Sx) is a function /i : Sx — > [0, oo] such 
that 

1. /u(0) = 

2. if {Ai\i G /} is a pairwise-disjoint family of measurable sets, with / countable, then 

If we have a measure taking values in [0, 1] we call it a sub-probability measure and if the measure 
("mass") of the whole space is 1 we say that it is a probability measure. A u-field equipped with a 
measure is called a measure space and equipped with a probability measure it is called a probability 
space. 

Sets of measure zero play an important role. The phrase almost everywhere is frequently used 
to assert that a certain property holds everywhere except on a set of measure zero. If there is 
confusion about which measure is intended we might say, for example, P-almost everywhere. 

The set of real numbers and the closed unit interval [0, 1] play a central role in the subsequent 
discussion. As measurable spaces, each has two cr-fields which are often used, the Borel cr-field 
and the Lebesgue cr-field. Any collection of subsets of a set X generates a a-field, namely the least 
(7-field containing all the sets of the given collection. If we take the open sets of any topological 
space and generate a a-field we get the Borel cr-field. In particular we get the Borel cr-field on 
the reals. This cr-field on the reals can be given a measure in such a way that the measure of an 
interval is its length. The resulting measure space has the property that there are subsets of sets of 
measure zero that are not measurable. There is a canonical "completion" procedure which yields 
an extended cr-field and measure, such that any previously measurable set has the same measure 
and all subsets of sets of measure are measurable (and have measure 0). When applied to the 
Borel subsets of the reals with the Lebesgue measure one gets Lebesgue measurable sets (with the 
Lebesgue measure). In our discussion we always mean Borel measurable whenever we talk about a 
measurable subset of the reals. 

In some older books p^ , a measurable function from the reals to the reals is defined to 
be a function where the inverse image of an open set has to be a Lebesgue measurable set rather 
than a Borel measurable set. This has the unfortunate effect that the composite of two measurable 



functions need not be measurable. A suitable reference for the above discussion is [44|, but any 



good book on probability theory such as Ash M, Billingsley [ttl], or Dudley [05[ covers this material. 
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7.2 A category of stochastic kernels 

Probability theory has been examined in the past from a categorical perspective. For example, 
Giry |32] has given the following construction, based on hints in unpublished notes of Lawvere. 
Wendt has examined this construction extensively |57, 

Let Meas denote the category of measurable spaces and measurable functions. We will now 
describe a triple T on the category Meas. In what follows, when we talk about measurable functions 
into [0, 1], we always mean the Borel c-field on [0, 1], denoted B. If {X, S) is an object of Meas, 
then we define T{X, S) to be the set of probability measures on {X, S) equipped with the least 
(T-algebra making the evaluations 

eB:T{X) [0, 1] defined by esiP) = P{B) 

measurable, where B ranges over the measurable sets of X. T acts on maps by the formula: 

T{f){P){B') = PU-\B')) 

where f:X and B' G Ey . 

The unit for the triple r]:id^T is defined by the formula: 

Vx{x){B) = xb{x) 

where x £ X and XB is the characteristic function of B. 

The multiplication fi:T^ ^ T is defined as follows. If P' € T^(X), then P' defines a measure 
on T{X), and we use it to form the following integral: 



fix{P')iB) = [ esdP' 

JT{X) 

With these definitions, one can then prove |]3^ : 

Theorem 7.3 {T,r],fj,) form a triple on Meas. 

To understand the structure of the Kleisli category, we require the following definition. 

Definition 7.4 If (X, S) and {X',T,') are measurable spaces, then a stochastic kernel on X x X' 
is a function 

p:X X S' ^ [0, 1] 

that is measurable in its first argument, for each fixed measurable set and a probability measure in 
its second argument for each point in X. 

Stochastic kernels are closely related to regular conditional probability distributions 0, 

If T is a stochastic kernel on X xY and p is a stochastic kernel onYxZ, then we can compose 
p and r to obtain a stochastic kernel t o p:X x Tiz [0, 1], using the following formula: 

r o p{x, C): J /)(-, C)dT{x, -) for allxe X,C e^z 
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Note that in the above formula p{—,C) is acting as the measurable function, and r(x,— ) as the 
measure. The associativity of this composition follows easily from the monotone convergence the- 
orem. 

So we obtain a category Stoch, whose objects are measurable spaces, and whose morphisms 
are stochastic kernels. The identity for this category is given by the (5-formula: 

Ai' A\ — ) ^ if X e A 
I otherwise 



One can now derive [32|: 



Theorem 7.5 The Kleisli category for the triple T is equivalent to Stoch. 

Given a morphism f:X^ TY in the Kleisli category, one obtains a stochastic kernel via the 
formula: 

F: X X Sy ^ [0, 1] is defined by B') = f{x){B') 
7.3 Probabilistic Relations 

While the category Stoch allows valuable insights into probability theory - for example, the 



Chapman-Kolmogorov equation is simply functoriality |32] - it lacks some of the structure one 
requires of a category of relations; notably the ability to take the converse. To pass to a category 
which is more relational in nature, we will use measures on the product space. Unfortunately one 
cannot compose measures in any simple way. Given measures on the product space, there is no 
obvious sense in which one can integrate them to compose as in the category Stoch. The idea is to 
rely on a basic theorem which says that given such product measures, on suitable spaces, one can 
construct a pair of stochastic kernels - which, together with the marginal distributions, determine 
the original measure on the product space - and then compose them in the manner described for 
Stoch. 

We now give the details of the construction. First suppose that we have a pair of measur- 
able spaces (X, Sx) and (y, Ey), a probability measure Px on (X, Sx), and a stochastic kernel 
h{x,B):X X Sy [0,1]. Then we have a unique measure P on the product such that for all 



P{AxB)= [ h{x,B)dPx{x) 

J A 



Thus if we have a pair of stochastic kernels h : X x Sy — > [0, 1] and k : Y x Tix — > [0, 1] and 
probability distributions Px on (X,Tix) and Py on (y, Sy) - satisfying an evident compatibility 
condition - we can reconstruct a unique probability measure on the product space. 

Conversely, given a measure P on the product X x y we can construct a measure on each of 
the factor spaces by setting Px{A) := P{A x Y) and Py{B) := P{X x B). These are called the 
marginals. Knowing one of the marginals and the appropriate stochastic kernel is equivalent to 
knowing the product measure. Clearly the pair of stochastic kernels does not uniquely determine 
the product measure; it does not even determine the marginals. We now need to show how to go 
from the product measure to the stochastic kernels. 
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The situation we have is: a pair of measure spaces {X, T^x, fJ-x) and {Y, Sy, /xy) and a measure, 
say a, on the product space equipped with the product cr-field, Sx tX" Sy- We want to construct a 
stochastic kernel h : X x Sy — > [0, 1]. The product space is a product in the category Meas and 
is equipped with the usual projections vri and 7r2 to X and Y respectively. We want to construct 
h : X X Sy — > [0, 1] as in the diagram 

X xY 





X 



Y 



such that 



h{x,B)iJLx = Oi[A X B). 



where h is the morphism (of the category Stoch) that we are trying to construct and 7ri,7r2, 
the projections, are morphisms of the category Meas. However, this construction requires some 
assumption on the spaces involved. 

More precisely, we require that the spaces are Polish spaces^. Recall that a Polish space is the 
topological space underlying a complete separable metric space. This assumption is quite common 
in probability theory and allows the construction of regular conditional probability distributions 0, 
14, We will not invoke these general concepts here. 

We state a slightly more general theorem from which the construction of h in the preceding 
paragraph follows immediately. 

Theorem 7.6 Suppose that (U, P) is a probability space, V is a Polish space with the Borel cr- 
field, written Sy, and {W, S^y) is a measurable space. Suppose that f is a measurable function from 
U to V and that g is a measurable function from U to W . Then there exists a Stoch morphism, 
i.e. a stochastic kernel, Q : W — > V as shown in the diagram 



U 



f 



V 




w 

such that for all A G Siy, B G Sy ; 

/ Q{giu),B)dPiu) = P{g-\A) n f-\B)). 

This Q is unique in the sense that if Q' is another stochastic kernel satisfying the same equation 
then for P-almost all u Q{u, •) and Q'(U, •) are identical. 

■^We could have more general spaces, for example analytic spaces |pq|. 
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Roughly speaking, this says that Q composed with g agrees with / at least when evaluated on the 
measures P. In probability texts this theorem is stated in terms of existence of regular conditional 
probability distributions relative to a sub a-field. We have essentially the same situation since 
the set of inverse images under g of the VF-measurable sets forms a sub-a- field of With this 



identification, theorem 7.6 is equivalent to theorem 10.2.2 of pq 



We are now ready for the corollary of chief interest. 

Corollary 7.7 Given Polish spaces X and Y with their Borel a-fields and a probability measure 
a on the product space, there is a stochastic kernel Qi{x,B) (i.e. a Stoch morphism from X to 
Y), where B £ Sy and a stochastic kernel Q2iy,A) (i.e. a Stoch morphism from Y to X), where 
A G Sx, such that 

I Qi{x, B)dax = a{A X B) = / Q2{y,A)daY- 

J A Jb 



Proof. We use the theorem 7.6 with XxYasU,XasW and y as ^ and the projection maps 



as / and g. Now we immediately get Qi. To see that the equation is satisfied we check as follows: 

a{TT^^{A) n TT^^iB)) = a{A x B). 



On the other hand the left hand side of the equation asserted in theorem 7.6 is, in this case, 

Qi{TTi{{x,y)),B)da. 



AxY 



This can be rewritten as 



/ Qi{x, B)da o TT^^ = / Qi(x,B)dax 
J A J A 



I A J A 

which is the desired result. One gets the result for Q2 similarly. I 

Here are two simple example applications of corollary [7.7[ For the first we take the product 
measure a tohe n^i^. In this case the stochastic kernel h : X x Sy — > [0, 1] is h{x, B) = v{B), i.e. 
it is independent of x. If we take the product X x X with the measure A defined by A(j4 x B) = 
Ijl{A r\ B), we get the usual Dirac delta 5{x, A). 

Finally, to define morphisms in our category, we proceed as follows. Given two measures, fi and 
u, on a measurable space we say v is absolutely continuous with respect /i, written << /i, if for 
any measurable set A, fJ.{A) = implies that I'^A) = 0. We now assume that the marginal ax is 



absolutely continuous with respect to fi. By applying the Radon-Nikodym theorem |14], we obtain 
a measurable function h{x):X R such that 



axiA) = / h{x)d^{x) 
J A 

From which it follows that: 

Q{x, B)dax{x) = / Q{x, B)h{x)diJ,{x) 
J A 



We refer to the function F{x,B) = Q{x,B)h{x) as the stochastic kernel associated to a. 



28 



Definition 7.8 We define a category PRel as follows. The objects of PRel are triples (X, 
where X is a Polish space, S the associated a field and /u is a probability measure on (X, S). 
A morphism a: {X, S, — > (X', E', //') is a probability measure on S S' whose marginals are 
absolutely continuous with respect to ^ and /j,'. 

To compose morphisms a: X ^ Y and y ^ Z, we calculate their associated stochastic 
kernels F{x, B) and G{y, C) and compose as in the above Kleisli category to obtain a stochastic 
kernel H{x,C). We then obtain a measure on X x Z via the formula: 

-f{A xC)= f H{x,C)dfi{x) 

J A 

Theorem 7.9 PRel is a category. 

Proof. The only thing remaining to consider is the identity. If (X, S,//) is an object, its identity 
is given by x A') = /x(An A'), with the associated conditional distribution given by the Dirac 
5. I 



Theorem 7.10 PRel is a tensored ^-category. 

Proof. The *-structure of PRel is evident, and the tensor product on objects is given by the 
product in the category Meas, that is, one takes the product of the 2 sets, the tensor of the a- 
algebras, and the product measure. The necessary equations are all straightforward to verify. I 

It is worth understanding the nature of isomorphisms in PRel in order to get a better sense of 
the role of the measures on the PRel objects. We consider first objects with the same underlying 
Polish space and hence cr-field. We will show that two such objects are isomorphic exactly when 
they define the same ideal of sets of measure zero. 

Proposition 7.11 Consider two PRel objects Xi and X2 where Xi = (X, E,/i) and X2 = 
(X, S, u). They are isomorphic in PRel if and only if fj, « v and v « /i. 

Proof. Suppose first that ^ « v and v « fj,. We define an isomorphism H : Xi — > X2 and 
K : X2 Xi as follows^. We set H{A x B) = fi{AnB) and K{A x B) = iy{AnB). The marginals 
are 

Hi = H2 = /X and Ki = K2 = v- 

By the absolute continuity assumptions these are PRel morphisms. The associated stochastic 
kernels are just the Dirac delta distributions and the composite of these distributions are again 
Dirac delta distributions. As we have observed before the Dirac delta distribution is the stochastic 
kernel associated with the identity morphism. Thus H and K form an isomorphism. 

Conversely, suppose that we have an isomorphism K : X\ — > X2 and K : X2 — > Xi. Suppose 
that n{A) = for some A £ T,. Let h' be the stochastic kernel from X2 to Xi associated with H, 
then we have 

/ h'{x, A)diy{x) = H{A X X) = Hi{A) = 

JX2 

*As usual we define measures on product spaces by specifying them on the semi-ring of "rectangles" and then 
relying on the standard extension theorems |14] to obtain the unique extension to the whole space. 
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where the last equahty follows from Hi « fx as required for to be a PRel morphism. We are 
writing integrals over X2 and Xi rather than over X in order to avoid confusion; of course Xi and 
X2 are both X as sets. Since h' is always nonnegative we have that it is i/-almost everywhere 0. Let 
k be the stochastic kernel from Xi to X2 associated to K. Since H and K form an isomorphism, 
we have 

/ h'{x',A)k{x,dx') = 5{x,A). 

JX2 

Integrating both sides of this equation over Xi using z/, we get 

f [f h'{x\A)k{x,dx')]du{x) = I 5{x,A)dv{x) =v{A) 

It can easily be shown, using the monotone convergence theorem, that we can rewrite the left hand 
side as 

/ h'{x\A)[[ k{x,dx')diy{x)] 

J X2 ^ X\ 

where the integral in square brackets defines the measure used for the outer integration. This 
measure is absolutely continuous with respect to v since it is defined by k. Since the integrand 
h'{x',A) is z^-almost everywhere 0, the whole integral is 0. Thus ^{A) = and v « ^. Similarly 



Observation 7.12 Similarly, given two Polish spaces and a Borel isomorphism between them, one 
can show that the two objects are isomorphic if and only if the Borel isomorphism preserves and 
reflects sets of measure zero. 



In view of proposition 7.11| and observation 7.12 the following important theorem of classical 



measure theory (see, for example, theorem 13.1.1 in |2^) almost completes the analysis of isomor- 
phisms in PRel. 

Theorem 7.13 If X and Y are Polish spaces, then X and Y are Borel isomorphic if and only if 
X and Y have the same cardinality. Moreover this cardinality must be either finite, countable or 
the cardinality of the continuum. 

Now we can state the main theorem about isomorphisms in PRel. 

Theorem 7.14 Let X and Y be two objects in PRel. Then X and Y are isomorphic if and only 
if there is a Borel isomorphism between them and that isomorphism preserves and reflects sets of 
measure 0. 



Proof. In view of theorem 7.13, it remains to show that isomorphic objects in PRel always have 
the same cardinality. First note that for finite or countable objects in PRel the stochastic kernels 
are just stochastic matrices. Thus an elementary rank argument suffices. 

In the case that one of the objects has an uncountable underlying set we argue as follows. It is 
easy to see that in an uncountable set, with any a-field and with any probability measure, say P, 
there can be at most countably many points, x, with P{{x}) ^ 0. 

Now suppose that (X, Sx, A*), with X a countable set, and {Y, Ey , i/), with Y uncountable, are 
PRel objects. Suppose, for the moment, that is nonzero for every x ^ X. Now suppose 
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that we have an isomorphism H : X — > Y with inverse K : Y — > X. Thus we have stochastic 
kernels as follows: 



h+,k- : X X Sy ^ [0,1] and : F x Ex ^ [0,1]. 



Since these are isomorphisms, we obtain the equation 




{x,B)h-{y,dx) = 5{y, B). 



Since X is countable, this reduces to 



{x,B)h-{y, {x}) = 5{y,B). 



Let B = {y}, where {y} is a set with z/-measure zero. Now observe that k must satisfy 



/ k-{x,{y})dii = K{Xx{y}) = KY{{y})=0 
Jx 



where the last equality is a consequence of the absolute continuity requirement. But 



k {x,{y})d^i=Y^k {x,{y])ii{{x]) 



xex 



By assumption, for every x E X, we have that ^({x}) ^ 0. Thus, for every x E X, it is the case 
that k~{x, {y}) = 0. So we conclude: 



This is a contradiction. 

Finally, recall that the stochastic kernels are uniquely defined only almost everywhere. In 
particular, for a countable probability space, the set of all points of measure zero itself has measure 
zero. Thus, at points where = 0, we can define k^{x,B) to be 0, and the above argument 



7.4 A nuclecir ideal for PRel 

To determine a nuclear ideal for PRel, we must consider the set Hom{I,X (8) Y). By definition, 
this consists of measures a which are absolutely continuous with respect to the product measure 
IJ, X By Radon-Nikodym, we can construct a measurable function f:XxY—^R such that for 
all C G Sx «) Sy: 



As usual two measures are equal if and only if their associated functions agree almost everywhere. 

Thus, we will define M{X, Y) to be the set of all measures on X x Y for which there exists a 
measurable function / such that the previous formula holds. It is immediate that the marginals 
associated to such a measure are absolutely continuous with respect to ii and respectively. While 



k~{x, {y})h-{y, {x}) = ^ S{y, {y}). 



xex 



still applies. 
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/ itself is only unique almost everywhere, the measure with which / is associated is easily viewed - 
in a canonical way - both as a member of Hom{X, Y) and as a member of Hom{I, X xY). Thus 
every element of the set Hom{I, X (g) y) is associated with a measure that has a functional kernel 
which is in turn one of the members of the set M{X, Y). 

To see that we have a 2-sided ideal, suppose that a G M{X,Y). Hence we have a function 
f-.XxY^R satisfying the above equation. Suppose (3 G Hom{Y, Z). let G2: Sy x Z — > R be the 
associated stochastic kernel. Then we define a function /i: X x Z — > R by the formula: 

h{x,z) = / f{x,y)G2{-,z) 

JY 

As usual, we are viewing /(x, y) as a measurable function of y for the fixed x, and G2(— , z) as 
a measure on Y for the fixed z. The construction for right composition is essentially identical. One 
can readily verify that the functions so constructed are indeed functional kernels for the composite 
measures. 

Finally, we observe that in the case when both a and (3 are nuclear, then there exist functions 
f{x, y) and g{y, z) which act as functional kernels. The functional kernel of the composite is given 
by: 

/ f{x,y)g{y,z)dn{y) 

JY 

We conclude: 



Theorem 7.15 The above construction determines a nuclear ideal for PHel. 

The verification of the requirements for a nuclear ideal are routine. The calculations involve 
computing transposes and can be done just the same way as proving associativity of composition 
in Stoch. We call this nuclear ideal MRel. One can generalize the setting to analytic spaces |25| 
which are continuous (or measurable) images of °° in Polish spaces. 



8 Trace Ideals 

In [^], Joyal, Street and Verity develop an abstract theory of trace operators in a monoidal category. 
A trace is a function of the form: 

trA- Hom{A, A) Hom{I, I) 

satisfying appropriate equations. (In fact, the authors introduce a more general parametrized trace 
which we discuss below.) The authors demonstrate that in a symmetric (in fact, braided) compact 
closed category, one obtains a trace via the formula (using the notation of section 2 and using c to 
represent the symmetry): 

{h: A ^ A) {iy;h<^ id; c;ip:I^ I) 

For example, in the compact closed category of finite-dimensional Hilbert spaces, one obtains 
the usual notion of trace of an endomorphism. This notion of trace also underlies such ideas as 
feedback in a computation and braid closure 



32 



When one passes from the category of finite-dimensional Hilbert spaces to the category of 
arbitrary Hilbert spaces, one finds endomorphisms which do not have a trace, for example the 
identity on an infinite-dimensional space. However, each endomorphism monoid contains an ideal 
of endomorphisms which do have a trace. This ideal is called the trace class and these trace maps 
are closely related to Hilbert-Schmidt morphisms. After reviewing this relationship, we describe a 
general theory of trace ideals for symmetric monoidal categories. We then show that if a tensored 
*-category has a nuclear ideal satisfying certain additional structure, then one can recover a trace 
ideal, as in the compact closed case. 

8.1 Hilbert Spaces 

Appropriate references for this material are p9| , 

Definition 8.1 An operator B £ C{H)-, the space of bounded linear operators on "H, is called 
positive if {Bx, x) > 0, for all x £7i. In this case, we write > and B > A ii A — B > Q. 

Note for example that AA* and A* A are always positive. 

Theorem 8.2 page 196) Suppose A>Q. Then there exists a unique B > such that B"^ = A. 

Definition 8.3 The unique operator B of the previous theorem is denoted ^/A. Let A £ C[H). 
Define = yjA*A. 

Theorem 8.4 Let TC be separable and {e-j} an orthonormal basis. If A is a positive operator, 
we define tr{A) = ^{Aen,en) ■ This is independent of orthonormal basis. It has the following 
properties: 

• tr{A + B) = tr{A) + tr{B) 

• tr{\A) = \tr{A), for all X>0 

• IfO<A<B, then tr{A) < tr{B). 

Definition 8.5 An operator A is called trace class if tr(|j4|) < oo. The family of all trace class 
operators is denoted by I{Ti.) or just I. 

Theorem 8.6 I has the following properties: 

• I is a vector space. 

• It is a 2-sided ideal in the monoid Hom(TC,TC). 

• If AG I, then A* e I 

These last two conditions say that we have a *-ideal. We now extend the notion of trace to 
arbitrary endomorphisms in the trace ideal. 
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Theorem 8.7 (f^, p. 211) If A £ I and {ej is (XTi ovthoTLOTuicil bcisis, then y~]^^-| (Acn,^ 6n) con- 
verges absolutely and is independent of the basis. (We call this map the trace of A, tr{A).) 

Using the notion of trace class, it is possible to give an equivalent formulation of the notion of 
Hilbert-Schmidt map: 



Proposition 8.8 (j^, p. 211) A mapping f:H^ICis Hilbert-Schmidt if and only if f*f G ^CH). 
The converse of this observation is also true: 



Proposition 8.9 (i4^], p. 211) If h is a bounded linear operator on 7i, then h G I if and only if 



there exist Hilbert-Schmidt operators f and g on 7i such that h = fg. 

Remark 8.10 Let 7i be a Hilbert space, and suppose we consider H as a Banach space. Then 7i is 
an object in the category Banoo, where we consider Banoo with its usual Li tensor product. Thus 
we can apply Grothendieck's original definition of nuclear morphism, and we see that we recover 
precisely the trace class maps. 

8.2 Trace Ideals 

The previous discussion suggests the following abstract definition. We suppose for the remainder 
that C is a symmetric monoidal category. 

Definition 8.11 A trace ideal in C is a choice of subsets 

I{U) C Hom{U, U) for each object U in C 

and a function 

trjj:Z{U) — > Hom{I,I) for each U in C 

such that 

1. X([/) is a 2-sided ideal in the monoid Hom(U, U). 

2. (Dinaturality or Sliding) Suppose that /:[/—> F and g:V ^ U are such that gf G T(U). 
Then fg e I{V), and tru{gf) = tryifg). 

3. ( Vanishing) If / G T{U), then / ® id/ G Z{U I) and tru(^j{f id/) = tru{f). Furthermore, 
we require that T{I) = Hom{I, I), li f : I I, then trj{f) = f. 

4. {Tensor Axiom) If / G I{U) and g G I{V), then f ® g £ 1{U ® V) and tru^vif ^ g) = 
tru{f)trv{g). 

5. Furthermore, if the category has a tensored *-structure, then we require that trace maps are 
closed under tensored *-structure, and the trace operators respect this structure, i.e. 

• If / G J(C/), then so is /*, and tr//(/*) = tru{f)* . 
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. If / G J([/), then / G 1{U) and trjj{f) = truif)*. 

An alternative approach to partial traces is presented in [jl8| , which considers traces on a linearly 
distributive category. The trace operator works on a certain subcategory, the core, which has the 
same sort of "type degeneracy" as a compact closed category. 

We would like to extend the relationship between compact closed categories and traced monoidal 
categories to a relationship between nuclear ideals and trace ideals. Keeping in mind the corre- 
spondence between Hilbert- Schmidt maps and the trace class, we define: 

Definition 8.12 Suppose that C is a tensored *-category equipped with a nuclear ideal. Suppose 
also that A is an object in C. We define the trace class of A to be: 

I{A) = {h: A ^ A\ There exists an object B, and morphisms f: A ^ B, g: B ^ A 

with f, g nuclear and h = gf} 
More generally, given two objects A,B £ C, one can define: 

I{A, B) = {h: A ^ B\ There exists an object C, and morphisms f: A ^ C, g: C ^ B 

with f, g nuclear and h = gf} 



Lemma 8.13 I{A) is a 2-sided ideal in the monoid Hom(A, A). I{A,B) is a 2-sided ideal in C. 

While one can define the notion of trace class for arbitrary morphisms in C as above, note 
that the actual trace function only acts on I (A) = I {A, A). In other words, the trace function 
acts only on the diagonal of the functor X(— ,— ). This is analogous to the notion of dinatural 
transformation, which is the appropriate notion of naturality for multivariate functors. These are 
families of morphisms between the two given functors, instantiated along the diagonals, satisfying 
an appropriate commutative hexagon |2^, 24, 12, Hence the alternate name "dinaturality" for 
the sliding axiom. 

If /i € we would like to define a morphism tr^(/i):/ — > / (or just tr{h) if there is no 

confusion) by the formula (where g,f denote the evident transposes): 

tr{h) = gf:I ^A(S) B ^ I 

However, there is no guarantee that if h is also equal to f'g' that we will obtain the same trace. 
Therefore we make the following definition: 

Definition 8.14 A nuclear ideal is traced if it satisfies the following uniqueness property: 

• If f:A^ B,g:B ^ A,f':A^ C,g':C ^ A are nuclear and gf = g' f , then gf = g'f': I ^ I. 



Theorem 8.15 The above construction assigns a trace ideal to each traced nuclear ideal. 

The proof of this theorem is simply a matter of checking the necessary diagrams. For example, 
lemma 5.8 gives the sliding axiom. One can also check that: 

Theorem 8.16 The canonical nuclear ideal in Hilb is traced. 



35 



8.3 Traces in DRel 

We now examine the trace construction in our category of distributions. 
Theorem 8.17 The canonical nuclear ideal in DRel is traced. 

Proof. Suppose that f: X ^ Y, g:Y ^ X, f: X ^ Z,g': Z ^ X are nuclear and gf = g' f ■ Since 
/ is nuclear, we have a morphism f: I ^ X ®Y , which has associated to it fi'-T^il) — > T>{X x Y). 
As already remarked, is isomorphic to the base field, hence the map fi simply picks out an 

element of D^X x Y), which we denote by (3f. Similarly for f',g,g'. 
To verify the uniqueness property, recall that if (/> E 'D(X), then 



Jx 

Since gf = g'f, we have that for (j)i,(p2 G ^i^)' 

fL{(t>l)9R{^2) = I /1(</'i)5r(</'2) 
Y Jz 

After rearranging the order of integration one can conclude: 

(3fix,y)(3g{y,x')(j)i{x)(j)2{x') = / / (3f>{x,z)(3gi{z,x')(j)i{x)(j)2{x') 

XJXJY JxJxJz 

The left-hand side corresponds to the distribution on X x X with kernel Jy f3f{x,y)f3g{y, x'), and 
the right-hand side has kernel f3fi{x,z)f3g'{z,x'). We know that two integrable functions induce 
the same distribution if and only if they are equal almost everywhere, but since these are smooth 
functions on X x X, we conclude: 



Thus we have: 



I3f{x,y)(3g{y,x') = / (3f>{x,z)f3g>{z,x') 

Y Jz 



^f{x,y)l3g{y,x) = / (3f'{x,z)(3g'{z,x) 
x Jy Jx Jz 



And we conclude tr{gf) = tr{g' f). 



Actually, there is a more succinct description of the trace operator in DRel. Since h = gf is 
nuclear, it has a kernel, a{x,x'). Recall from theorem 6.13| that the formula for a is given by: 



Y 



a{x,x') = fR{l3g{y,x')) = l^(3f{x,y)Pg{y,x') 
Hence we may conclude that: 

trA{h) = I oi{x, x) 



X 



We leave the details of the following to the reader. The result is quite similar to the case of 
DRel. 

Theorem 8.18 The canonical nuclear ideal in PRel is traced. 
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8.4 The parametric trace operator 



In [40 1, the authors actuahy have a parametrized trace operator. This means that there is a function 
of the form: 

tru: Hom{A (g)U,B ^U) ^ Hom{A, B) 

which reduces to the usual trace when A = B = I . There is an evident generahzation to the ideal 
setting: 

Definition 8.19 We suppose again that C is a symmetric monoidal category. A (parametric) trace 
ideal in C is a choice of a family of subsets, for each object C/ of C , of the form: 

1a,b ^ Hom{A (g)U,B(^U) for all A,B in C 

and functions 

trls-^ls^ Hom{A,B) 
such that the families are ideals in the sense that: 

• If / G "^AB /i: is arbitrary, then {id /i) o / and / o (id h) are in 1^^- 

• If / G g: B ^ C, h: D ^ A are arbitrary, then {g ® idjj) o / o (/i idu) G (j. 

These are subject to the ideal-theoretic versions of the Joyal-Street- Verity axioms. In particular, 
(dropping sub- and superscripts if there is no chance of confusion) 

• (Vanishing) 

1. ^ = Hom(A I,B I), and the trace is calculated in the evident way. 

2. Suppose g:A(g)U0V^B(g)U(g)V. Then g G if and only if 5 G lAm,Bm and 
t^Am,Bm^9) e Furthermore, 

trTB{9)=trlj,{tr\^U^j,^u{9)) 



(Superposing) Suppose / G and g:C^Dis arbitrary. Then g ® f ^ '^c®A d®b^ ^^d 
tr{g® f)=g®tr{f). 

(Yanking) Suppose f: A ^ U and g:U ^ B. If cc/,_b ° (/ "SD ff) £ ^a B^ then 

trlB{cu,Bo{f(^g))=gf:A^B 

(Shding) Suppose f:A(^U^B(^V and u:V ^ U. Then {id®u)o f ^ if and only if 
/ o (id (8) n) G 1a B^ ^^d the two traces are equal. 

(Tightening) Suppose / G Iab and g: B C,h: D A are arbitrary. Then 

tr({g (gi idfj) o / o (/i (g) idu)) = 9 ° t'f'if) ° h 
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• Furthermore, if C is a tensored *-category, then the trace must preserve this structure in an 
evident sense. 



Some discussion of our version of the Yanking axiom is in order. The Joyal-Street- Verity 
version of this axiom is essentially the requirement that the trace of a symmetry morphism is the 
identity. However, in our framework, one cannot make this requirement since the symmetry map 
will generally not be in the trace class. In the forthcoming thesis of Haghverdi p5|], it is observed 
that the following requirement is equivalent to the Joyal-Street- Verity version: 

Generalized Yanking Rule: 

Suppose f:A—>U and g:U B. Then, 



8.5 [/-nuclear ideals 

As before, we would like to construct trace ideals from nuclear ideals. An analogous construction 
can be carried out using the notion of a U -nuclear ideal. We now outline this idea, but leave most 
of the details to the reader. The generalization amounts to introducing the notion of a U -nuclear 
morphism. We will say that a morphism f:A®U^B is [/-nuclear, if it has a transpose f: A ^ 
U ® B. More specifically, for each object U , we introduce a family of morphisms Nu{A ®U,B) C 
Hom{A® U,B). These families should be closed under all of the operations and furthermore an 
ideal in the sense that if 



tr%{cu,Bo{f(S)g))=9f:A^B 



f eAfu{A^U,B) 



and h:V ^ U is arbitrary, then 



{{id^hyj) eArv{A^v,B) 



Similarly for the variables A and B. 

Also there should be a natural bijection of the form: 



Afv{A ^V,B)^ Ny{B V, A) 



satisfying appropriate equations. For example, the compactness requirement becomes: 



• (Compactness) Suppose f: A ^ C ® B and g: B ® D ^ E . Then we have: 



A®D 



f ® ido 



C®B®D 



idA ® g 



idc0g 



A®B®E 



f ®idE 
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If a tensored *-category is equipped with such structure, we will refer to it as a parametrized 

nuclear ideal. 

Given such a construction, one defines the U -trace classIu{A®U,B®U) C Hom{A(^U, B^U) 
by saying that: 

heIu{A^U,B^U) 

if and only if there exist 

f eJ\fuiA0U,C),g e Mu{B ® U, C) such that h = g*f 

One then constructs the U-tiace of h via the formula: 

tr^^B{h)--A^C®U ^ B 

where the components are the evident transposes of / and h. Again, one must add conditions to 
ensure that the trace satisfies appropriate equations. In particular, we note that with the above 
axioms, we can only obtain the following weaker version of the yanking axiom: 

Lemma 8.20 Suppose that C is a tensored ^-category equipped with a parametrized nuclear ideal. 
If f: X ^ U and g:U^Y are nuclear, then c o (f g): X U ^ Y U is in the U -trace class, 
and 

trSc,Y{co{f^g))=gf 
This is a consequence of the compactness requirement of section 5. 



8.6 Traces in PInj 

We now discuss the traced structure of PInj. First it is evident that unlike in Hilb, we have that 
X(A) = M{A, A) for all objects A. li f: A ^ A is a trace map, then we have the following formula: 



tr{f) 



id if \Dom{f)\ = 1, and / is the identity when restricted to its domain. 
otherwise 



The parametrized trace also has a very simple description. We will say that a morphism 
/: X (g) [/ — y is [/"-nuclear if it satisfies: 

Vx G X if {x,u) G Dom{f) and {x,u') G Dom{f), then u = u' 

Given this definition, there is an evident bijection J\f{X ®U,Y) = J\f{Y fS> U, X). 
The class I{X fS> U,Y U) is described by having the above requirement for both the domain and 
codomain. Then we can say that if / G Tr{X ®U,Y ®U), {x, u) G Dom,{f) and f{x, u) = (y, u'), 
then: 



undefined if u u' 
V ii u = u' 
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9 Conclusions 



Our investigations began with an attempt to define probabilistic relations in analogy with ordinary 
relations. Unexpectedly, ideas from functional analysis [33| were essential. The key idea, expressed 
in our abstract definition of nuclear ideals, is that certain morphisms can be thought of as behaving 
like "matrices". 

Our work naturally follows on from the development of Higgs and Rowe |37|, the fundamental 
difference being that we have no closed structure. Crudely speaking, Higgs and Rowe generalize 
Banach space theory while we generalize Hilbert space theory. 

A key application of our work is that we can now work with structures that are not categories 
but which are nuclear ideals inside some tensored *-category. For example, the nuclear ideal MRel, 
described in Section 7, is of interest but is not a category. (As an example of its possible applications, 
we note that MRel has partially additive structure |^, ^.) However, MRel is indeed a nuclear 
ideal in PRel. 

An important open question is the computational significance of trace ideals. It is already 
well-established that a trace structure can be used to model feedback in denotational semantics 
1 40, SC]. But what can be said when one only has these operations on an ideal? The geometry 
of interaction program, due to Girard |31], can be used to obtain a compact closed category from 
a traced monoidal category [^, ^, 4C]. It seems possible that a similar construction applied to a 
category with a traced ideal will give a nuclear ideal. 

Another area of application of the theory of compact closed categories is topological quantum 
field theory ^, which evolved, in part, from Segal's work on conformal field theory ||5^. In 
topological quantum field theory, one considers a compact closed category of cobordisms in which 
composition is defined by gluing along boundaries. Then a TQFT is given by a compact closed 
functor to the compact closed category of finite-dimensional Hilbert spaces. In Segal's formulation 
of conformal field theory, one works with arbitrary Hilbert spaces and a similar "category" of 
Riemann surfaces with boundary. This structure is essentially a compact closed category, except 
that it fails to be a category in that it lacks identity morphisms. Thus it seems reasonable to 
suspect that it is a nuclear ideal in some larger ambient tensored *-category. One of our goals 
in future work will be to find such a category. A conformal field theory would then be a nuclear 
functor to the tensored *-category Hilb. 

A related issue is the extension of our work to higher-dimensional categories. The theory of 
n-Hilbert spaces |11|, a higher-dimensional analogue of Hilbert space, has become quite important 
in TQFT M]. Baez has developed the theory of 2-Hilbert spaces with this in mind, and extended 



some of the work of Doplicher and Roberts to this setting [23 



Finally, the category DRel suggests several further topics of investigation. One possible ex- 
tension of DRel is to the theory of noncommutative distributions |^]. Roughly speaking, these are 
distributions which take values in a Lie group. They are useful in the representation theory of 
gauge groups. Finally, we hope to take advantage of the fact that distributions form a V-module, 
that is to say they provide representations of the Weyl algebra [^] . It would be interesting to at- 
tempt to extend the work of |16, 17], where full completeness theorems are obtained by considering 
representations of the additive group of integers and a noncocommutative Hopf algebra. 
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